The development of audiovisual speech perception

The seemingly effortless and incidental way in which infants acquire and perfect the ability to use spoken language/s is quite remarkable considering the complexity of human linguistic systems. Parsing the spoken signal successfully is in fact anything but trivial, as attested by the challenge posed by learning a new language later in life, let alone attaining native-like proficiency. It is now widely accepted that (adult) listeners tend to make use of as many cues as there are available to them in order to decode the spoken signal effectively (e.g. Cutler 1997 ; Soto-Faraco et al. 2001 ). Different sources of information not only span various linguistic levels (acoustics, phonology, lexical, morphology, syntax, semantics, pragmatics, etc.), but also encompass different sensory modalities such as vision and audition (Campbell et al. 1998 ) and even touch (see Gick and Derrick 2009 ). Thus, like most everyday life perceptual experiences (Gibson 1966 ; Lewkowicz 2000a ; Stein and Meredith 1993 ), spoken communication involves multisensory inputs. In the particular case of speech, the receiver in a typical face-to-face conversation has access to the sounds as well as to the corresponding visible articulatory gestures made by the speaker. The integration of heard and seen speech information has received a good deal of attention in the literature, and its consequences have been repeatedly documented at the behavioral (e.g. Ma et al. 2009 ; McGurk and MacDonald 1976 ; Ross et al. 2007 ; Sumby and Polack 1954 ) as well as the physiological (e.g. Calvert et al. 2000 ) level. The focus of the present chapter is on the developmental course of multisensory processing mechanisms that facilitate language acquisition, and on the contribution of multisensory information to the development of speech perception. We argue that the plastic mechanisms leading to the acquisition of a language, whether in infancy or later on, are sensitive to the correlated and often complementary nature of multiple crossmodal sources of linguistic information. These crossmodal correspondences are used to decode and represent the speech signal from the first months of life.